Method selection for species distribution modelling: are temporally or spatially independent evaluations necessary?

To assess the realism of habitat projections in the context of climate change, we conduct independent evaluations of twelve species distribution models, including three novel ecosystem-based modelling techniques. Habitat hindcasts for 24 western North American tree species were validated against 931 palaeoecological records from 6000, 11 000, 14 000, 16 000 and 21 000 yr before present. In addition, we evaluate regional extrapolations based on geographic splits of  55 000 sample plots. Receiver operating characteristic analyses indicated excellent predictive accuracy for cross-validations (median AUC of 0.90) and fair accuracy for independent regional and palaeoecological validations (0.78 and 0.75). Surprisingly, we found little evidence for over-parameterisation in any method. Also, given high correlations found between model accuracies in non-independent and independent evaluations, we conclude that non-independent evaluations are eff ective model selection tools. Ecosystem-based modelling approaches performed below average with respect to model sensitivity but excelled in specifi city statistics and robustness against extrapolations far beyond training data, suggesting that they are well suited to reconstruct historical biogeographies and glacial refugia.